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The purposes of this paper are to assess the National Assessment 
of Educational Progress's (NAEP) survey of reading achievement/ and to 
offer some recommendations for its agenda in the future. The paper 
consists of three parts. In Part 1/ some of the key results from the 
four national assessments of reading are described. Part 2 contains a 
description of NAEP 1 s objectives for its reading assessments over that 
same time period/ and discusses some assumptions that NAEP has made. 
In Part 3, recommendations are made for ways in which NAEP might 
revise its future assessments. 
NAEP's Reading Results/ 1970-84 

After four national assessments of reading achievement by NAEP, 
several trends in reading achievement have emerged. Among then* are 
the following: 

(1) The reading achievement of the nation's 9-/ 13- 7 and 17-year 
olds has increased from 1971 to 1984 (see Table 1). Gains 
for minority students have been particularly large, although 
the gap between their reading achievement and the national 
level continues to be great. 

(2) The students who have made the most gains are those who were 
born in 1965 - Black children in this cohort have made the 
greatest improvement of all, although in 1984/ Black 
17-year olds were reading at the same level as White 
13-year olds. 

(3) Of the three categories of reading skills that NAEP assesses 
(i.e./ literal comprehension / inferential comprehension/ and 
reference skills)/ gains within each age group have been 
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greatest for reference skills (see Table 2). Reference 
skill exercises are designed to measure students 1 ability 
to find and use resource materials. Based on the exercises 
NAEP has released/ those assessing reference skills appear 
to require the least amount of reading of the three 
categories. 

(4) In spite of continued improvements in the reading 
achievement of all three age groups across the four 
assessments/ the actual achievement level of many of the 
nation's students appears to fall below that required for 
meeting the demands of grade appropriate reading materials . 

(5) Variables such as level of parental education/ amount of 
reading material in the home/ extent of television watching/ 
and amount of homework done are all related to students' 
reading achievement . 

Overall/ it appears that we have learned a great deal about the 

nation's levels of reading achievement from NAEP's assessments. In 

this regard/ NAEP "works". To assess how well it works and how it 

might be improved/ however/ some additional questions must be asked: 

What were NAEP's goals in its assessment of reading achievement? Have 

these goals been met? 

NAEP's Objectives and Assumptions/ 1970-1984 

From its inception, NAEP has consisted of a set of objectives. 
The purpose of these objectives has been to provide NAEP with content 
validity, the only kind of validity with which it is concerned. 
NAEP's goal is that each of its exercises measure "some important bit 
of knowledge or some important skill that reflects one or more of the 
objectives" (Womer, 1970, pg. 9). Thus, NAEP's content validity is 
established when "it 'makes sense' to an informed reader who sees 
together an objective and an exercise designed to measure that 
objective" (Womer, pg. 9). 
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NAEP's intention is that its objectives reflect the current goals 
and practices in the subject area being assessed. As a result/ NAEP 
is a dynamic rather than a static assessment. From its beginning/ 
NAEP expected that its objectives might change from one assessment to 
the next/ and objectives have been reviewed/ a^d when necessary 
revised/ before each new assessment. 

A listing of NAEP's objectives for its four assessments of 
reading is contained in Appendix A. As can be seen from looking at 
these objectives/ changes have indeed occurred/ although it is a 
change in number more than kind that is most apparent . For example/ 
NAEP used 137 objectives to guide its exercise development in the 
first reading assessment/ whereas only 13 were used in the most recent 
one. Thus, one of the ways that NAEP's conception of the goals and 
practices in reading has changed has been to move toward more global 
categories of knowledge and skills. 

In looking at changes in the content of the objectives from year 
to year/ many of the changes that occurred seem to mirror the shift in 
reading theory and research from a focus on "product" to one on 
"process". For example, in the 1970-71 assessment, one category of 
objectives was concerned with assessing students 1 ability to "read 
phrases, clauses, and sentences". In the 1979-80 objectives, this has 
been changed to "comprehends propo.si tional relationships". 

Despite NAEP's move toward more global and process-oriented 
objectives, a general consensus seems to have existed from one 
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assessment to the next about those aspects of reading that are 
essential to measure. Central to the goals and practices of reading, 
according to NAEP's objectives, has been students 1 ability to arrive 
at different levels and kinds of understanding and response to 
different levels and kinds of text. 

Interestingly, what is not apparent from looking at NAEP's 
reading objectives throughout the years are the changes that have 
occurred in reading instruction during this same time period. For 
example, since NAEP began, an increased emphasis in reading 
instruction has been placed on phonics as a means for improving 
students' reading ability (e.g., see Chall, 1983a). However, none of 
NAEP's changes in reading objectives seem to reflect this change in 
practice. It even appears that NAEP has made a move away from 
assessing the skills and knowledge that would be most directly 
affected by phonics instruction in its more recent objectives. Thus, 
NAEP's objectives may be better reflections of prevailing views about 
reading than they are of tho goals and practices in reading 
instruction . 

Another important feature of NAEP's assessment of reading emerges 
when we compare its objectives for assessment with what it seems to 
assess. For example, Appendix B contains the categories of exercises 
NAEP has used to report ths reading results from each assessment. 
Recalling that the purpose of the objective? is to provide NAEP with 
content validity, it is interesting how little correspondence there 
seems to be between each assessment's objectives and the categories 
that NAEP has used to analyze and report the results. 
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From the reading exercises that have been released (e.g./ see 
Reports No. 02-R-20 and ll-RL-25), it is apparent that NAEP did 
directly tie exercises to objectives in at least two of its 
assessments (i.e., 1970-71 and 1978-79). However, in it summaries of 
its ?4'-*Mlts# NAEP has always seemed to opt for categories that differ 
from those described by the objectives. 

Before turning to some recommendations for ways in which NAEP's 
reading assessment can be made more useful for practitioners, 
researchers, and decision-makers, one more qeneral characteristic of 
NAEP needs to be described: the theory of reading and reading 
development that underlie the assessment. 

Consistent with classic as well as current approaches (e.g., see 
Samuels & Kamil, 1984; Singer & Ruddell, 1985), NAEP is based on the 
view that reading consists of both lower level skills (e.g., word 
identification, knowledge of word meanings, sentence understanding, 
etc.) and higher level skills (e.g., inference making, drawing 
conclusions, etc.). Thus, NAEP includes in its assessment of reading 
a wide variety of exercises designed to assess how well students 
perform on these skills. In addition, NAEP assumes that as reading 
ability develops, there is an increase in the need to be proficient in 
higher level skills. Again, this assumption is consistent with 
established views on reading development (e.g., see Chall, 1983b), and 
is reflected by NAEP's use of increasingly challenging reading 
materials and exercises as the age of the students being assessed 
increases. 
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In interpreting its results/ NAEP goes on to make one further 
assumption: that when students fail to comprehend challenging reading 
materials, this failure stems from a lack of proficiency in higher 
level skills rather than from any deficiencies in lower level ones. 
This last assumption is an important one for two reasons: first, 
because it has been the basis for NAEP's conclusion that although 
students' proficiency in lower level skills has been improving, 
increased instructional efforts need to be directed toward improving 
higher level ones ; and second , because of all the assumptions tha t 
NAEP makes, it may be this one that has the weakest theoretical and 
empirical support • 

In terms of theory, most models view reading as a process that 
always requires lower level and high level skills, regardless of the 
level of reading development (e.g., see Chall, 1983b) or the 
difficulty of the material being read (e.g., see Pearson, 1984). As 
reading ability develops, or the level of difficulty of reading 
materials decreases, lower level skills such as word identification do 
seem to require less and less effort on the reader's part to complete. 
However, even when lower level skills are performed by the reader 
without any conscious effort (i.e., they are automatic), lower level 
skills still remain a critical component of reading. 

NAEP ' s asessment of reading , on the other hand , tends to assume 
that students' proficiency in lower level skills can be assessed with 
relatively simple reading tasks , whereas level of proficiency in 
higher level skills is what is being indicated by their performance on 
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more challenging reading tasks. This assumption is probably best 
illustrated by the scale that NAEP has used to summarize the results 
from the most recent assessment. 

NAEP's new scale consists of five levels of reading proficiency/ 
ranging from rudimentary to advanced. Such a scale allows NAEP to 
make comparisons across age groups and across assessments, as well as 
to describe the kinds of reading tasks that students at different ages 
are able to do. 

So, for example, in reporting the results of the 1983-84 
assessment, NAEP concluded that while 95% of the nation's 17-year olds 
have "the ability to understand specific or sequentially related 
information" (i.e., they have reached a "basic" level of reading 
proficiency), less than 40% of these students are able to "find, 
understand, summarize, and explain relatively complicated information" 
(i.e., they have not reached an "adept" level of proficiency). What 
NAEP does not say, but appears to be the case from the "benchmark 
exercises" that NAEP provides to anchor its proficiency scale, is that 
95% of 17-year olds were able to deal with materials written at a 
4th - 5th grade level, while less than 40% were able to understand 
materials written at a 7th grade level. 

Why the majority of 17-year olds in 1983-84 had difficulty with 
7th grade materials is not clear from NAEP's results, however. It may 
be, as NAEP suggests, that they do lack the "adept skills and 
strategies" that materials at this level of difficulty require. If 
this is the ca#>f . then NAEP would seem to be on target in its call for 
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an increased emphasis on higher level skills for these students. It 
may also be, though/ that these students are experiencing difficulty 
in performing at a "basic" level when the level of difficulty of 
materials is increased from 4th to 7th grade. If this is the case/ 
then increased opportunities for instruction in lower level skills 
would seem to be essential for these students as well. 

Interestingly , data collected in the past by NAEP itself helps to 
further clarify this issue. For example, in the 1970-71 assessment, 
NAEP used "reading themes" to analyze and summarize the reading 
results, a theme being a category that "defines a set of existing and 
potential exercises that relate to each other in content or in some 
central idea that is meaningful to the subject area of concern" (NAEP/ 
1974a/ pg. 6). The eight reading themes used by NAEP (the 1970-71 
assessment) are shown in Appendix B. 

NAEP conceived of these themes as hierarchical in nature/ such 
that those at the beginning of the list were viewed as lower level 
skills (e.g./ "Understanding words and word relationships")/ while 
those toward the end were viewed as higher level (e.g./ "Reading and 
drawing inferences" ) . 

Although the results from the 1970-71 assessment were summarized 
in terms of "group median effects" (i.e./ the median difference of a 
group from a national performance level)/ it is possible to calculate 
students 1 percent correc t on exercises wi thin each theme from other 
information that NAEP has released (i.e./ NAEP/ 1973). Table 3 
contains students 1 percent correct on three of the themes: Theme 
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1--Understanding words and word relationships; Theme 5 — Gleaning 
significant facts from passages; and Theme 7 — Reading and drawing 
inferences. 

As can be seen from this table/ students at all age levels in 
1970-71 tended to be no more successful on lower level skills than 
they were on higher level ones. 

Comparable analyses from later assessments are not possible , 
since NAEP no longer used "themes" to analyze and report results after 
1970-71. "However/ from the exercises released after the 1979-80 
assessment (NAEP, 1981c), it is possible to calculate students 1 
percent correct on categories of exercises that seem somewhat 
comparable — i.e., exercises designed to assess the following three 
objectives: Comprehends words and lexical relationships; Comprehends 
proposi tional relationships; and Comprehends textual relationships. 
These results are shown in Table 4. 

Again , as indicated by the resul ts on the released exercises , 

little difference seemed to exist in students' performance in the 

1979-80 assessment on those exercises designed to assess lower level 

skills and those designed to measure higher level ones. And yet, it 

was after this assessment that NAEP's report on the results concluded 

the following: 

. . .The downward trends in reading of 13- and 
17 -year- olds , particularly in the area of 
inferential comprehension , are signaling 
deteriorating resources and instruction for 
those higher- order intellectual abilities that 
go beyond basic skills. If these trends 
continue into the 1980's then it seems plausible 
that we are failing to give these students 
anything but basic skills (NAEP, 1981a, pg. 48). 
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It must be pointed out that this conclusion by NAEP was based/ 
not on the data contained in Table A, but on the changes that occurred 
in students' performance from 1971-1980 on exercises classified as 
"inferential" ones (i.e./ see Table 2). Notice/ in addition/ though/ 
that these trends seem to have changed again between 1980 and 1984/ 
suggesting that perhaps reading instruction may have changed in the 
direction recommended by NAEP/ or that NAEP may have misinterpreted 
those earlier changes. 

One of the goals for NAEP/ from its beginnings/ was to stimulate 
debate and further research (Womer/ 1970). uno of the purposes of 
this paper/ in describing NAEP's results/ otjo* tives / and 
interpretation of the reading data collected over the last four 
assessments/ has been to suggest that many aspects of the assessment 
itself point to the need for both. 

The purpose of the final section of this paper is to look toward 
future assessments/ however/ rather than evaluating the past. Its 
premise is that NAEP can revise its next assessments of reading in 
ways that would place debates and research on interpretations on more 
solid ground. 

Recommendations for Future Assessments in Reading 

The "Idea" of National Assessment was to develop 
a plan for gathering direct information about 
knowledges/ about understanding/ about skills - 
information not currently available... If 
Nationa 1 Assessment provides information that can 
be helpful in making wiser educational decisions/ 
it will ' ave achieved its goal/ its "Idea". 
National Assessment is the "Idea" that accurate 
information about what boys and girls are learning 
is an essential ingredient for wise decision-making 
in education. Information alone does not change 
education; people who use information wisely can 
change education, (pg. 2-3 ) . 
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The following recommendations are offered in the spirit of the "Idea" 

of NAEP / as described above by Frank Womer in 1970. The 

recommendations fall into thr' 2 sets : Set 1 contains some suggestions 

for changes in the information NAEP provides about the data it 

currently collects; Set 2 contains some suggestions for additional 

data that NAEP should collect; and Set 3 contains some suggestions for 

changes in the way that NAEP has come to define itself. 

Recommendation 1: NAEP needs to provide 
more complete information about the kinds 
of reading tasks that the nation's 9-,- 13-/ 
and 17-year old students can and cannot do. 

The first set of recommendations is in response to NAEP's addition/ in 

the 1983-84 assessment/ of a reading proficiency scale. The scale's 

purpose is to allow direct comparisons to be made across age groups/ 

and from one assessment to the next. In these ways/ it is an 

important change in NAEP/ since it greatly facilitates our ability to 

identify age differences/ as well as changes within age groups across 

time. In other ways, however, the scale has the potential to move us 

farther away from the "Idea" of NAEP than we have ever been before. 

Whereas in previous assessments/ we needed to consider changes in 

percent correct on a known number of exercises that were categorized 

by types/ now we are faced with considering changes in scores that 

range from 0 to 500 on an unknown number of exercises measuring what 

we are not sure. To aid us in interpreting these new scores/ NAEP has 

provided us with one or two exercises at five different levels on the 

scale/ ranging from 150 to 350. And/ although these "benchmark" 

exercises are of some help/ NAEP still needs to provide us with more 

information that will allow us to distinguish between descriptions of 
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students' reading ability and NAEP's interpretations of its results of 
how well students read. 

l.A. Fetter anchors need to be provided 

for NAEP's reading proficiency scale. 

Since the validity of NAEP rests solely on its content (rather than 
how well it predicts success at something else/ for example)/ NAEP 
needs to provide us with more information about each of the levels of 
"reading skills and strategies" that it uses to report its results. 
In its summery report (NAEP, 1985) NAEP tells us what percentage of 
each age group are able to accomplish reading tasks at each level. 
One rather simple addition would be to provide us with an example of a 
reading exercise given to each age group from each level. For 
example, NAEP found that in terms of rudimentary skills and 
strategies/ "across the four assessments, virtually all of the 13- and 
17-year olds and most of the 9-year olds were able to accomplish such 
reading tasks" (NAEP, 1985, pg. 17). And yet, the exercises that NAEP 
provides us with as anchors at this level of proficiency are exercises 
that were given only at age 9. If NAEP could also provide examples of 
exercises that "virtually all of the 13- and 17- year olds" could do, 
we would have a much better idea of what a rudimentary level of 
reading skills and strategies means, and what NAEP's results tell us 
about the nation's 13- and 17-year olds. 
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NAEP also suggests that "it is the relationship between the 

complexity of the passage and the way in which the reader needs to go 

about finding the answer to a particular question that shapes the 

demands of a reading task" (NAEP, 1985, pg. 14). And yet, from the 

descriptions and examples provided for each proficiency level, it is 

difficult to ascertain the nature of that relationship. Sample 

passages clearly increase in complexity as proficiency level 

increases; however, the change in questions does not appear to be as 

straightforward, or to conform as well to NAEP's description of what 

each level of proficiency requires. Perhaps, if possible, NAEP could 

describe each level of proficiency in terms of not only what students 

are able to do, but also what they can not. Appendix C gives an 

example of a passage from the 1978-79 assessment that illustrates how 

this could be done. Whereas 70% of 17-year olds were abl' 

correctly answer the first exercise (requiring recognition of a 

paraphrase from a relatively complex passage), only 31% of the 

students understood the purpose of the entire passage (exercise 2). 

In terms of NAEP's levels of proficiency, exercise 1 seems to require 

"intermediate reading skills and strategies," while exercise 2 seems 

to require an "adept" level. 

l.B. NAEP needs to provide more information 
about the number and kinds of exercises 
that are used to assess each level of 
proficiency. 

In addition to roviding better anchors for its scale, NAEP needs to 
tell us more about the exercises that are used as the basis for the 
scale. For example, although NAEP tells us that few items fell at the 
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ends of the continuum of the 500 point scale , it would also be helpful 
to know how many items fall at ether points along the scale, and how 
many items are actually good discriminators between proficiency 
levels . 

In addition, NAEP has always included three kinds of exercises in 
its reading assessments: literal comprehension, inferential 
comprehension, and reference skills. Although NAEP states that it was 
primarily the literal and inferential questions that were used as a 
basis for the new proficiency scale, it needs to be more specific 
about the numbers of each kind of exercise that were included. In the 
past, improvements in students' reading ability seem to have stemmed 
largely from improvements in performance on the small number of 
reference skill items that NAEP has included (see Table 2). 
Understanding the results on changes in average proficiency levels 
across assessments requires that we know whether this remains the 
case • 

Recommendation 2 : NAEP needs to collect 
information that tells us not only about 
the kinds of reading tasks that students 
can and cannot do, but also about the kinds 
of knowledge and skills that underlie their 
performances. 

The previous set of recommendations was concerned with improving 
NAEP's description pf the kinds of information it currently collects. 
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The purpose of this set of recommendations/ however/ is to describe 
some ways in which NAEP could collect some additional information that 
would improve the usefulness of the reading assessments. 

Students 8 proficiency in reading at all levels of 
development is affected not only by how well they deal with the 
"message" aspects of what they have read/ but also by their ability to 
deal with the "medium" itself (Adams, 1980; Chall, 1983b). Cui/rently, 
however/ NAEP makes no attempt to distinguish between students 1 skills 
in each of these aspects of reading. As a consequence/ although NAEP 
provides us with some information about the kinds of reading tasks 
that students can and cannot do/ we have little or no information 
about the sources of their successes and failures, 

NAEP has 7 in the past/ collected information that allowed it to 
report on students 1 lower level and higher level skills (see section 
on "NAEP 1 s Objectives and Assumptions"). Several factors make the 
argument for reinsti tuting this practice a strong one. 

First/ as discussed earlier/ NAEP has always assumed that 
students 1 failure on exercises designed to assess complex 
comprehension strategies and skills stems from a lack of proficiency 
in higher level skills. However/ as has already been pointed out, 
this is not always the case . As reading materials increase in 
difficulty/ so do vocabulary load and sentence length (e.g./ see 
Chall, 1984). As a result/ reading materials that require higher 
level skills for understanding also place increased demand on lower 
level skills * By assessing students 1 proficiency in both the lower and 
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higher level skills required by reading materials along a continuum of 
difficulty/ NAEP would be providing essential information about the 
sources of students' successes and failures. 

Second/ information about the status of students' lower level and 
high level skills could be invaluable for educational decision making. 
As NAEP (1985) has pointed out, increased attention to higher level 
reading skills has already begun in schools across this country. 
Other changes in reading instruction (such as the increased emphasis 
on phonics mentioned earlier , the increased difficulty of basal 
readers, etc.) have occurred as well (see Chall, 1986). With NAEP 
providing us with better information about the kinds of knowledge and 
skills underlying students' reading performances, we would all be in a 
much better position to evaluate the need for particular instructional 
approaches. 

Finally, NAEP has long been encouraged to find ways in which its 
results can become more useful to states and local school districts 
(e.g., see the 1976 report by the U.S. Comptroller General). NAEP 
could provide more detailed geographical breakdowns for the 
information it currently collects, allowing comparisons to be made 
between a state and the nation, or between states. However, there is 
reason to question how useful this kind of information would be. 
NAEP's reading proficiency score is so general that it is difficult to 
say what practical s ignif icance there would be in knowing that 
students in a particular state score 5 points above or below the 
national average. 
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If, however, NAEP were to collect information that provided a 
clearer piccure of students 1 reading knowledge and skills, states and 
local school districts would be in a better position to use any 
information released to them. In addition, trends found in data 
already collected at local and state levels could more easily be 
compared to those found at the national level. 

If jJAEP is to provide better information about students' reading 

ability, decisions will have to be made about the kinds of data to 

collect. Currently? from what NAEP tells us, it appears that it 

already has several exercises designed to assess higher level skills* 

NAEP could use performance on these items to construct a scale of 

higher level reading skills, similar to the more global scale that it 

already uses. It is unclear, however, how many exercises NAEP has 

that assess lower level skills at each age level. As a result, the 

following suggestions are directed toward describing ways that lower 

level skilla could be more directly assessed. 

2. A. NAEP needs to provide information 
about the level of knowledge that 
9-, 13-, and 17-year old students 
have about word meanings. 

Several reasons exist for why NAEP should include a scale of 
vocabulary knowledge as part of its reading assessment. Knowledge of 
word meanings is a major factor in reading comprehension (Davis , 
1968), as well as one of the best single predictors of success in 
school (Carroll, 1971; Terman, 1918). Furthermore, vocabulary tests 
provide us with a good estimate of the extent of students 1 background 
knowledge , or "schemata " , a factor that plays an important role in 
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students 1 success in understanding and learning from what they have 
read (see Anderson & Pearson, 1984). Finally, vocabulary knowledge is 
a component of reading that virtually everyone seems to agree is 
important. Disagreements abound about how vocabulary is best taught 
(e.g., see McKeown & Curtis, 1987), but knowledge about word meanings 
has long been a well acknowledged part of the goals and practices in 
reading. 

In the past, NAEP has included vocabulary exercises in its 
reading assessments, presenting words both in a context and out. In 
Appendix D, sample exercises from the 1979-80 assessment are shown. 
Although each exercise was preceded by a reading passage, the sentence 
in which each word appeared is a part of each exercise. 

Since NAEP already includes vocabulary exercises as part of its 
reading assessment, why should it include a separate scale? 
Vocabulary knowledge (like reference skills) differs from other 
aspects of reading that NAEP assesses. With the exception of some 
words that have multiple meanings, students' performance on these 
exercises does not depend on their ability to comprehend texts. 
Students' performance on exercises that require text comprehension, 
however, can depend on the knowledge of word meanings that they bring 
to the reading task. Because vocabulary exercises are more likely to 
be "passage independent " than comprehension exercises , a vocabulary 
scale would inform us about the extent of knowledge students bring 
with them to reading tasks. 
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A separate scale for vocabulary knowledge would also allow us to 
track vocabulary development, both across and within age groups, from 
one assessment to the next. Comparisons among levels of vocabulary 
knowledge, proficiency in higher level skills, and NAEP's more general 
scale would also aid us in understanding why students are able or 
unable to do certain kinds of reading tasks. 

In designing ways to describe levels of students' vocabulary 
knowledge, NAEP should consult sources such as word frequency lists 
(e.g., Carroll, Davies, & Richman, 1971) and graded vocabulary lists 
(e.g., Dale & O'Rourke, 1981). This would ensure that the vocabulary 
exercises measure knowledge of word meanings that are important for 
students at differing age levels to know. Since the exercises need 
only present a sentence context (see Appendix D) , NAEP would no longer 
need to restrict the vocabulary it assesses to those words that are 
contained in the passages it uses. 

Finally, NAEP should consider the benefits of assessing students 1 

word knowledge through listening as well as reading. A reading 

vocabulary task requires that students know the meanings of words. 

However, it also requires that students have the ability to identify 

those words in print, a demand not made by a listening task. By 

assessing students' vocabulary knowledge in both modes, comparison 

between results on each would allow NAEP to provide important 

information about students 1 strengths and needs in reading (see 

Carroll, 1977). 

2.B. NAEP needs to provide information 
about the efficiency wi th which 
students are able to deal with text. 
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Vocabulary assessment/ particularly when done through both the 
reading and listening modes, can tell us much about students' ability 
to deal with texts at varying levels of difficulty. In addition to 
students' knowledge about word meanings/ however/ a large body of 
research also strongly supports the need to understand how efficiently 
students are able to deal with the texts they must read. 

Reading efficiency refers to the ease with which students are 
aole to identify accurately the words in a text (e.g./ see Lesgold & 
Curtis/ 1981; Perfetti/ 1985). When students are efficient at 
reading, their efforts are directed at the meaning (or "message") of a 
text. When students are not efficient at reading, however, dealing 
with word identification (or the "medium" of a text) requires effort. 
As a consequence, they are less able to deal with the meaning of a 
text. 

Reading efficiency is often estimated by measuring the rate at 
which students are able to read and understand what they have read. 
For example, college students read silently in the vicinity of 250-300 
wpm, whereas the nightly news is read to us at about 175 wpm (Sticht, 
1984; Sticht & James, 1984). 

In NAEP's first assessment of reading (1970-71), information was 
collected on students' reading rate and comprehension (see NAEP, 
1972). Students were given a passage to read that was judged 
appropriate for their grade level in school. Following the passage, 
students were asked five multiple choice questions that required word 
for word recall of details from the passage. Reading rate was 
recorded for each student. 
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Table 5 contains a summary of the results from that part of the 
assessment. The average reading rate for 9-year olds was 117 wpm; for 
13-year olds, 173 wpm; and for 17-year olds, 195 wpm. NAEP reported 
the results in 100 wpm intervals. Based on NAEP's results, as well as 
other research (see Sticht, 1984 for a summary), students in the right 
hand column of Table 5 should be considered slower than average 
readers . 

What is so interesting about these results is how the percentage 
of students who read efficiently (i.e., they are not slow readers and 
they are able to remember what they have read) declines across the age 
groups. Whereas 51% of the 9-year olds were efficient readers, only 
38% of the 13-year olds and 31% of the 17-year olds were. 

NAEP needs to reassess the efficiency with which students at each 

of these age levels deal with texts at varying levels of difficulty. 

If students' reading efficiency has improved over the last decade and 

a half, then that is important information for us to know. If reading 

efficiency has not changed, or declined, then we will have a much 

better idea about the sources of students' difficulties in dealing 

with the texts they must read. 

Recommendation 3 ; Until NAEP collects 
information about the ways in which 
reading is taught, it needs to avoid 
drawing impl i cat ions about the ways 
that reading should be taught. 

One of NAEP's goals has always been to collect information that 
will aid in making educational decisions. As Womer reminded us in 
1970: "Information alone does not change education; people who use 



information wisely can change education" (pg.3). Over the past two 
assessments^ however/ NAEP has increasingly succumbed to joining in 
the decision making process. In considering this move, NAEP needs 
also to consider some dangers that would seem to accompany it. 

First/ as Jeanne Chall (1986) has noted/ it is very difficult to 
say how instruction should be changed without first knowing what kind 
of instruction is already occurring. As Chall further notes, it can 
also be very risky. In NAEP's case, the difficulty and risk seem 
particularly great/ as interpretation of its results remains an open 
issue (e.g./ see Chall/ 1986; Curtis & Glaser, 1983; Venezky, 1977). 
If the decison is made that NAEP's mission is to include using its 
results to make instructional recommendations/ then two actions will 
be required: first/ NAEP will need to resolve this issue of 
interpretation, and second/ NAEP will need to collect information 
about school and classroom characteristics. Until these changes have 
occurred/ however/ it would seem that NAEP can no more say how 17-year 
olds should be taught reading than it can say how well they should be 
reading. 

The second danger that NAEP faces lies in the way that 
assumptions about how reading should be taught can affect decisions 
about what should be measured. NAEP's assessment of reading is 
supposed to be guided by the current goals and practices in reading. 
If/ however/ it becomes tied to certain hypotheses about what the 
future goals and practices in reading should be, it could begin to 
provide information that will not be useful/ or even worse/ erroneous. 
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Table 3 

1970-71 : Percent Correct on Exercises in Three Selected Themes 



9-year olds 13- year olds 17 -year olds 



Theme 1: 
Words & Word 

Relationships 75% 65% 68% 

(n=13) (n=19) (n=14) 



Theme 5: 

Gleaning Significant 

Facts 59% 70% 82% 

(n=25) (n=49) (n=42) 



Theme 7: 

Reading and Drawing 

Inferences 74% 60% 67% 

(n=37) (n=45) (n=38) 



ERLC 
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Table 2 

1971-1984: Changes in Percent Correct by Type of Question 



9-year olds: 

1971-1975 
1975-1980 
1980-1984 

Over 1 

13-year olds: 
1971-1975 
1975-1980 
1980-1984 

Overal 1 

17-year olds: 
1971-1975 
1975-1980 
1980-1984 

Overall 



Li teral 

1.0 
2.8 
-0.6 



3.2 

0.7 
0.9 
-0.9 



0.7 

0.5 
-0.7 
-0.7 



-0.9 



Inferential 

0.9 
2.5 
-1.1 



2.3 

-0.8 
0.2 
0.9 



0.3 

-0.9 
-1.2 
1.3 



-0.8 



Reference 

2.3 
2.6 
2.6 



7.5 

-1.7 
2.6 
1.6 



2.5 

0.6 
0.2 
2.9 



3.7 
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Table 1 

1971-1984: Changes in Students' Scores Over Assessments 



9-year olds: 

1971-1975 
1975-1980 
1980-1984 

Overall 

13-year olds: 
1971-1975 
1975-1980 
1980-1984 

Overall 

17-year olds: 
1971-1975 
1975-1980 
1980-1984 . 

Overall 
V Note: Proficiency 



Proficiency Scores 

2.4 
3.9 
-0.3 



6.0 



Percent Correct 

1.3 
2.6 
0.2 



4.1 



0.9 
2.6 
0.4 



3.9 



-0.1 
0.9 
0.4 



JL . *~ 



0.2 
0.0 
3.7 



3.9 

scores are based on a scale of 



0.0 
-0.8 
1.1 



0.3 

0 to 500 (NAEP / 19 



Table 4 



1979-80: Percent Correct on 



9-year olds 



Words 61% 

(n=17) 



Propositions 57% 

(n=15) 



Text 55% 

(n=14) 



Released Exercises for Objective II 



13-year olds 17-year olds 

77% 75% 

(n=24) (n=15) 

77% 80% 

(n=20) (n=16) 

70% 75% 

(n=21) (n=21) 
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Table 5 

1970-71: Results from Assessment of Rate and Comprehension 

100 wpm or faster Slower than 100 wpm 

9-year olds: 

80-100% recall 
0-60% recall 

13-year olds: 

80-100% recall 
0-60% recall 

200 wpm or faster Slower than 200 wpm 



51% 
13% 



22% 
14% 



38% 
53% 



2% 
8% 



17-year olds: 

80-100% recall 
0-60% recall 



31% 
15% 



36% 
18% 



Appendix A 

Reading Objectives for 1971-1984 Assessments 



1970-71 Assessment: Reading Objectives 



I. Comprehend what is read 

A. Read individual words (12 subob jectives listed) 

B. Read phrases , clauses , and sentences ( 12 ) 

C. Read paragraphs/ passages, and longer works (7) 

II. Analyze what is read 

A. Be able to trace sequences (3) 

B. Perceive the structure and organization of the 
work (6) 

C. See the techniques by which the author has 
created effects (8) 

III. Use what is read 

A. Remember significant parts of what is read (3) 

B. Follow written directions (2) 

C. Obtain information efficiently (20) 

IV. Reason logically from what is read 

A. Draw appropriate inferences (7) 

B. Arrive at a general principle after examining a 
series of details (3) 

C. Reason from a general principle to specific 
instances (3 ) 

V. Make judgments concerning what is read 

A. Relate what is read to things other than specific 
material being read (7 ) 

B. Find and use appropriate criteria in making 
judgments (27) 

VI. Have attitudes about and an interest in reading 

A. Depth of interest in reading (3) 

B. Motives for reading (7) 

C. Quantitative measures of reading interest (7) 



Note: Based on information contained in Reading Objectives 
(NAEP, 1970). 
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Appendix A, continued 



1974-75 Assessment: Reading Objectives 



I . Demonstrate behavior conducive to reading 

A. Demonstrate values related to reading (4 
subob jec tives ) 

B. Assess the readability of materials (2) 

C. Demonstrate knowledge of their own reading 
ability (3) 

II. Demonstrate word identification skills 

A. Know the letters of the alphabet 

B. Apply knowledge of sound symbol relationships 

C . Apply structural analysis techniques 

D. Possess basic sight vocabulary 

E. Use context for word identification 

III. Possess skills for reading comprehension 

A. Utilize written language conventions ( 2 ) 

B. Demonstrate literal understanding (3) 

C. Demonstrate inferential understanding (9) 

IV. Use a variety of approaches in gathering information 

A. Demonstrate flexibility in adapting reading rate 
to purpose and nature of materials (3) 

B. Possess reading study skills (2) 

C. Use reference materials efficiently (3) 



Note: Based on information contained in Reading Objectives : 
Second Assessment (NAEP / 1974). 
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Appendix A, continued 



1979-80 Assessment: Reading and Literature Objectives 



I. Values reading and literature 

A . Values the benefits of reading for the individual 
(3 subobjectives) 

B • Appreciates the cultural role of written 

discourse as a way of transmitting, sustaining 
and changing the values of a society 

II. Comprehends written works 

A. Comprehends words and lexical relationships 

B. Comprehends proposi t ional relationships 

C. Comprehends textual relationships 

III. Responds to written works in interpretative and 
evaluative ways 

A. Extends understanding of written works through 
interpretation (4) 

B. Evaluates written works 



IV. Applies study skills in reading 

A. Obtains information from nonprose reading 
facilitators 

B. Uses the various parts of a book 

C. Obtains information from materials commonly found 
in libraries or resource centers 

D. Uses various study techniques 



Note: Based on information contained in Reading and 
Literature Objectives (NAEP, 1980). 



Appendix A/ continued 



1983-84 Assessment: Reading Objectives 



I- Comprehends what is read 

A. Comprehends various types of written materials 

B. Comprehends materials read for a particular 

purpose 



II. Extends comprehension 

A. Analyzes what has been read 

B. Interprets what has been read 

C. Evaluates what has been read 



III. Manages the reading experience 

A. Uses the structure and organization of the text 

B. Uses readers 1 aids 

C. Shows flexibility in approach to reading 

D. Selects reading material appropriate to the 
purpose 

IV. Values reading 

A. Values reading as a source of enjoyment 

B. Values reading to expand understanding and 
fulfill personal goals 

C. Values reading as a means of acquiring knowledge 
and learning new skills 

D. Values the cultural role of written language 



Note: Based on information contained in Reading Objectives; 
1983-84 Assessment (NAEP / 1984). 
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Appendix B 



Categories Used to Report Reading Results 



Understanding words and word relationships 
Reading and visual aids 
Written directions 
Reference materials 

Gleaning significant facts from passages 
Reading for main ideas and organization 
Reading and drawing inferences 
Critical reading 



Literal comprehension 
Inferential comprehension 
Reference skills 



Literal comprehension 
Inferential comprehension 
Reference skills 



Rudimentary skills and strategies 
Basic skills and strategies 
Intermediate skills and strategies 
Adept skills and strategies 
Advanced skills and stra tegies 
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Appendix C 

Examples of Reading Exercises for 17-year olds (from NAEP, 1981c) 

Competition is healthy 

Last year the Supreme Court presented doctors/ lawyers/ 
dentists/- and other professionals with a right that most of them did 
not want - the right to advertise their services to the public. 
Since then the professions/ especially law and dentistry/ have been 
acrimoniously divided over the question of advertising. 

Older lawyers and dentists with established practices have 
spurned the idea of hawking their services/ as though they say/ they 
were selling another dog food or deodorant. But young men, trying to 
find a market for their services, have seized the opportunity to go 
to the public. Established members of the profession accuse them of 
misleading the public and undermining professional standards. 

It is easy to sympathize with someone who has built a practice 
the hard way and sees it threatened by an interloper who values the 
hard sell above professional dignity. But the fact remains that most 
of the professions could benefit from an injection of old-fashioned 
competition. In a world where fees are never publicized and the 
quality of work is hard to judge, the public has no way to tell 
whether it is getting its money's worth. 

If established practitioners think the public is being misled by 
irresponsible advertising, there is always a step they can take. 
They can advertise themselves/ both individually and through 
professional groups. They can tell the public what they think good 
practice is and what it should cost. They can describe the services 
they perform and what qualifications they have. 

If some professionals abuse the privilege of advertising/ there 
are plenty of laws on the books to bring them into line. But it is 
time for established practitioners to realize that the public needs 
to know more than a little bronze plate on the door can tell it. 

According to the editorial, what can older 
professionals do if they think the public is being misled by 
advertising? 

They can request that the Supreme Court reverse its decision. 

They can wait until the public tires of advertising by 

professionals. 

They can advertise their own qualifications and services. 

They can expel from the profession anyone who advertises. 



I don 1 t know. 
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What is the main purpose of the editorial? 

To explain the new law which allows advertising by 

professionals. 

To show the problems younger lawyers/ dentists/ and doctors 

have getting started. 

To encourage people to see the need for advertising 

professional fees and services 

To warn people about the dangers of advertising by 

professionals 

I don 1 t know. 



Note: 70% of 17-year olds in 1979-80 answered exercise #1 correctly; 
31% answered exercise #2 correctly. 
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Appendix D 

Examples of Vocabulary Exercises (from NAEP/ 1981c) 

The editorial says: 

It is easy to sympathize with someone who has built a practice 
the hard way and sees it threatened by an interloper who values the 
hard sell above professional dignity. 

What does the word interloper mean in this sentence from the 
editorial? 

17-year olds: 55% correct 

A person who does not believe in competition 

A person who intrudes upon others 

A person who misleads the public through advertising 

A person who is an established member of a profession 

The article says: 

There is the same empathy at the end of a marathon swim. 
What does the word empathy mean in this sentence from the article? 
17-year olds: 45% correct 

The joy of victory 

The pain in one's body 

A sharing of another's feelings 

An unforgettable experience 

I don' t know 
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The article says: 

They might ask a family from the "old country" to let them 
board with the family for a while. 

What does the phrase board with mean in this sentence from the 
article? 

9-year olds: 33% correct; 13-year olds: 80% correct 

Visit and work with 

Build a room with 

Pay to live with 

Learn English with 

The package directions say: 

Top with wet plate, invert the plate and mold together, 
remove mold 

What does the word invert mean in this sentence from the package? 

9-year olds: 26% correct; 13-year olds: 64% correct 

Turn upside down 

Remove 

Break apart 

Heat up 

The article says: 

In 1703 he built the city of St. Petersburg on the Gulf of 
Finland, hoping to open a new avenue to the West. 

What does the word avenue mean in this sentence from the article? 

13-year olds: 65% correct; 17-year olds: 76% correct 

Route 

Street 

Territory 

Trade 
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